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Abstract 

Using techniques developed in |Lasn2j . we show that some minimum cardinahty 
problems subject to linear inequalities can be represented as finite sequences of semidef- 
inite programs. In particular, we provide a semidefinite representation of the minimum 
rank problem on positive semidefinite matrices. We also use this technique to cast the 
problem of finding convex lower bounds on the objective as a semidefinite program. 

Keywords: semidefinite programming, sum of squares, rank minimization, K- 
moment problem, Lagrangian relaxation. 

Notation 

We note R[xi, a;„] (or R[x] when there is no ambiguity) the ring of multivariate polynomi- 
als p{x) = p{xi, ...,Xn) on a variable x G R"". We say that p{x) G R[x] is SOS when p{x) is a 
sum of squares of polynomials in H[x]. For x G R", Card(x) will be the cardinal of the set 
{i : Xi ^ 0}. We note S" the set of n x n symmetric matrices. For multivariate polynomials, 
we adopt the multiindex notation p{x) = YliaPaX", where := Xi^x'^'^ ...x'^" , and we note 
d = Yll^=i the degree of p{x). R^fa^] is the set of polynomials of degree at most d. Finally, 
C{p) will be the Newton polytope of the polynomial p{x), with C{p) = Co {{a : Pa 0}). 

1 Introduction 

Given a convex set C C R", we are interested in solving the following problem: 

minimize Card(a;) 
subject to x G C, 

in the particular case where C is described by a set of linear inequalities. Except in certain 
rare instances, this problem is very hard to solve (see |VB96j ). Excellent heuristics exist 



however, a classical one (see |HHB99j for example) replacing the function Card(x) by 



\x\ 



its largest convex lower bound on the unit cube. 
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A related problem is that of minimizing the rank of a p.s.d. matrix subject to LMI 
constraints: 

minimize Rank(X) , s 

subject to X G C, 

where C is here an affine subset of the semidefinite cone (a LMI). In this case also, minimizing 
the nuclear norm \\X\\^ of X will produce excellent approximate solutions (see FHBOOJ). 

I n this p aper, using results by [H^ . jEEEHZl, [P^ - IDLESE], [N^ . [L^ . 
and |Lasn2j . we show that the MinCard(x) and MinRank(X) problems in and Q 
are equivalent to large semidefinite programs (see |NN94j ) . To be precise, based on a refor- 



mulation a la jSho87j of problems (H)) and we use the technique in |Las02j to produce a 
finite (possibly exponential) sequence of increasingly tighter semidefinite relaxations. 

The rest of the paper is organized as follows. In section |2l we recall some key definitions 
and properties on semidefinite representability and the sum of squares representation of 
positive polynomials. We also summarize the application of these representations to semial- 
gebraic problems. In sectional we show that both the Min Card(a;) and the Min Rank(X) 
problems are equivalent to large scale semidefinite programs. Based on the work by [Put93j, 
INesOOj and |Lasn2j we explicitly construct in section lU a sequence of semidefinite programs 
solving problems and Q ■ We also show how the problem of finding optimal convex lower 
bounds on the objective function can be represented in a similar way. Finally, in section El 
we discuss the complexity of these techniques. 



2 Sums of squares and semidefinite programming 

We quickly recall here some key definitions and properties linking semidefinite and semial- 
gebraic problems. 

Hubert's 17*^ problem (see |Rez96j for an overview), which asked if all positive polyno- 
mials could be written as sums of squares of other polynomials, has a positive answer in 
dimension one. |NesOOj provides an efficient way of computing the SOS representation of a 
given positive univariate polynomial in the following result from |NesOOj . 

Proposition 1 Let p{x) G Il[x] be a univariate polynomial of degree d. Then p{x) for all 
X iff there exists a matrix X & such that: 

p{x) = y^Xux, with X ^ 0, for all x (3) 

with V = \d/2] and Ux = (l,x,a;^, ...^x'") is the list of univariate monomials up to degree v. 

The coefficients of the polynomials in the representation are then computed as the eigen- 
vectors of the matrix X. 

In the general multivariate case, that representation property of positive polynomials is 
lost. It can be shown (see |Ber80j ) that the set of multivariate SOS polynomials is dense in 
the set of positive polynomials, but there are simple examples of positive polynomials that are 
not SOS. However, recent results in semialgebraic geometry (see |Cas84j . |Sch91j . |Put93j 
or |PV99j ) bridge the gap between positive and SOS polynomials on compact semialgebraic 
sets. We cite here the result in |Put93j . Let gk{x) G R[xi, x„] for j = 1, r, and we note 
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the semialgebraic set defined hj K = {x E R" : gk{,x) > 0, /c = 1, r}. We suppose that 
K is compact and that there exists u{x) G R[a:i, x„] such that {u{x) > 0} is compact 
with 

r 

u(x) = Uo{x) + ''^^gk{x)uk{x), for all x e R"' (4) 

k=l 

where the polynomials Uk{x) G R[xi, x„] are SOS for k = 1, r. Under this assumption, 
we can represent all polynomials positive on K using SOS polynomials as in |iPut93j or 
jPV99| . 

Proposition 2 Suppose holds. A polynomial p{x) G R[a:;i, x„] is positive on K iff: 

r 

P{x) = qo{x) + ^gk{x)qk{x), for all x G R" (5) 

k=l 

where the polynomials qk{x) G R[xi, x„] are SOS for k = 0, r. 

Now, as in IParOOj or ILasOlj . we can write the multivariate version of the relation (jH)) 
mapping SOS polynomials to the semidefinite cone. 

Proposition 3 Let p{x) G R[xi,...,x„] he a polynomial and K a semialgebraic set defined 
by K = {x E R" : gk{x) > 0, k = 1, r}, satisfying assumption Then p{x) > on K 
iff there is an integer m G Z_|_ and matrices Xk G S^, with Xk ^ for k = 0, r such that: 

r 

Pi^) = ylXoVa: + ^ (yJXfc?/^) gk{x), for all x G R", (6) 
fc=i 

where N = [("'^™^^)/2'l and = (1, Xi, x^, XiXa, Xix„, X2X3, x^, x^^, x^) 
is the vector of all monomials in R[xi, ...,x„], up to degree m, listed in graded lexicographic 
order. 

The result on polynomials above shows that testing the positivity of a multivariate poly- 
nomial on a semialgebraic set K satisfying the assumption (j^ can be cast as a semidefinite 
program. In general, the result in |Las01j shows that all compact semialgebraic problems, 
i.e. problems seeking to minimize a polynomial over a compact semialgebraic set, are equiv- 
alent to large-scale semidefinite programs. This provides a positive answer in the compact 
multivariate case to all the open questions in §4.10.2 in |BTN01j . A converse result is also 
true (and much simpler). Because the positive semidefiniteness of a matrix is equivalent to 
that of all its principal minors, all semidefinite programs are semialgebraic programs, with 
additional convexity and invariance properties. 

The central result of moment theory exploited in [LasOlj sets polynomial positivity prob- 
lems and moment problems as duals (see e.g. jBerSOj ). Let s be a positive semidefinite 
sequence s G R^, we have 

s is p.s.d. 

t 

{s,Pa) > 0, for all p(x) G Rm[2;] with p(x) SOS, 
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and 



s is a moment sequence 

t 

{s,Pa) > 0, for all p{x) G Rm[a;] with p{x) > on R", 



hence the cone of coefficients of SOS polynomials and that of p.s.d. sequences are polar, 
and so are the cones of moment sequences and positive polynomials. 

From jPut93j then, we know that the problem of testing if a sequence y is the moment 
sequence of some measure /x with support in a compact semialgebraic set 



and the problem of representing positive polynomials on K as gk{x) weighted sums of SOS 
polynomials are dual of each other and both representable as linear matrix inequalities. 

3 Semidefinite representation of the MinCard(x) and 
MinRank(X) problems. 

As above K is the semialgebraic set defined hj K = {x & TV^ : gk{x) > 0, k = 1, ...,r} and 
we assume that (j3)) holds. Let p{x) G R[a;i,...,x„] and as K is compact, we also note 
t* = mmxpix). Of course, p{x) — t* > on K, hence there are SOS polynomials qk{x) G 
R[xi, Xn] for k = l,...,r such that © holds for p{x) — t* on K. We ffist show that 
the MinCard(x) problem can be cast as a semialgebraic program, hence a semidefinite 
program, using the results from section |21 

Proposition 4 Let A G R™^" and b G R"^. There are polynomials gk{x) G R[xi, 
for k = 0, ...,r such that the optimum values of: 



K = {xeR'':gk{x)>0,k = l,...,r} 



minimize Card(a;) 
subject to Ax > b, 



(7) 



and 



minimize go{x) 

subject to gk{x) y 0, for i = 1, . 



r 

1 ' 1 



are equal. 



Proof. First, as in |Sho87j we notice that: 



Card(x) = min X^iLi 

s.t. [vi — l)xi = 

Vi > 0, for i = 1 



n 



hence the Min Card(x) problem in ^ can be written: 



MinCard(x) 



En 
i=l ""i 

S.t. {vi — l)xi = 
Vi > 0, for i 
Ax > b. 



l,...,n 



which is a semialgebraic problem. ■ 
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We now show a similar result on the MinRank(X), a minimum cardinality problem on 
the eigenvalues of the matrix X. 

Proposition 5 Let Ai G S*^, for i = l,...,p and b G R^. There are polynomials gk{x) G 
R[xi, Xn], for k = 0, r such that the optimum values of: 

minimize Rank(X) 

subject to Tr (AjX) = 6j, for i = 1, (8) 
X ^ 

and 

minimize go{x) 

subject to gk{x) >z 0, for i = 1, M 

are equal. 

Proof. We note (Ai, A„) the eigenvalues of the matrix X ^ and 

o'fc(X) = Aq, 

a(Z{l,...,n},\a\=k 

the symmetric functions. We note xt (X) the characteristic polynomial of the matrix 
X, with Xt (X) = Yl'i=i <^i(^)^*- Because the matrix X is semidefinite positive, 
we have crfc(X) = iff Rank (X) < k, hence the MinRank(X) problem can be 
expressed as a minimum cardinality problem on the coefficients of the characteristic 
polynomial. With 

Rank(X) = min Vi 

s.t. ai{X){vi - I) = 

Vi > 0, for i = 1, n, 

we enforce the remaining constraints to get the following representation of Min Rank(X): 

minimize X^^Li '^j 

s.t. (vi - l)ai{X) = 

Vi > 0, for i = 1, n 

Tr{AiX)=bi, fori = l,...,p 

di{X)>0, for / C {l,...,n}, 

where di{X) is the principal minor with index set / C {1, . . . ,n}. This is a semialge- 
braic program in the coefficients of the matrix X. ■ 

These two results together with the results cited in section El show that the two problems 
considered are equivalent to very large scale semidefinite programs. 

4 Semidefinite relaxations 

In practice, the exact representations obtained in the last section can be exponentially large 
and in general, we cannot expect these problems to be tractable. Hence, the central con- 
tribution of these representations is not to reduce the complexity of these problems, but to 
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provide a sequence of successively sharper relaxations covering the entire complexity spec- 
trum, thus allowing the complexity/sharpness tradeoff to be tuned. This is what we intend 
to describe in this section. 

We begin by recalling the construction of moment matrices as detailed in [(yFOOj , [LasOlj 
and jLa.sn2]. Again, we let = x„, x?, X1X2, ...,xiXn,X2X3, ...,xl, be 

the vector of all monomials in R[xi, a;„], up to degree m, listed in increasing graded 
lexicographic order. We note s(m) the size of the vector y^. Let y G R**^^*") be the vector 
of moments (indexed according to y^) of some probability measure fi with support K = 
{x G R*^ : g{x) > 0}, we note Mmiy) G S^'-"*^ for the moment matrix defined by 

M^{y)ij = / (y^). (y^). fx (dx) , for i,j = 1, s(m) (9) 
Jk 

i.e. the (symmetric) matrix of moments with rows and columns indexed as in y^. We 
note j3{i) the exponent of the monomial {yx)i and conversely, we note i{j3) the index of the 
monomial x^ in y^. For a given moment vector y G R''^'^^ ordered as in Q, the first row and 
columns of the matrix Mm{y) are then equal to y. The rest of the matrix is then constructed 
following: 

Mm{y)i,j = ya+f3 if Mm{y)i,i = ya and Mm{y)j,i = yp- 

Similarly, let g{x) G R[a;i, a;„], we derive the moment matrix for the measure g{x)fx (dx) 
on K (called the localizing matrix), noted Mm{gy) G S'^'-™-', from the matrix of moments 

MUy) by: 

Mm{gy)ij= {yx)i{yx)jg{x)fi{dx) (10) 
Jk 

for i,j = 1, s{m). The coefficients of the matrix M„i{gy) are then given by: 

Mm{gy)i,j = ^gaMmiy)i(p{i)+p{j)+a) (11) 
a 

We can remark as in jLasOlj that if the measure /i has its support included in i^T = 
{x G R" : g{x) > 0}, then for all coefficient vectors v G R'"^™^: 

{v,Mm{gy)v) = / v{x)'^g{x)ij{dx) > 
Jk 

hence M^igy) h 0. 

In dimension one, for a given vector y G R'^'^"'\ M^{y) h (which is a LMI) is also a suf- 
ficient condition in order for y to the moment sequence of a probability measure. In R*^, this 
equivalence does not hold in general. The compact semialgebraic case is called the K-moment 
problem and is dual to the compact SOS problem in ©. Following [LasOlj . we now ex- 
ploit this duality to compute a sequence of semi definite relaxations for the Min Card (a;) and 
MinRank(X) problems. 
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4.1 The MinCard(x) problem 

In section 13 we saw that the optimum value of the Min Card(a;) problem can be computed 
as the optimum value of the semialgebraic program: 

En 

S.t. {Vi - l)Xi = 

Vi > 0, for z = 1, n 
aijX > bj, for j = 1, ...,m. 

As in |Las01j . to ensure compactness, we impose the additional constraint x1 + ... + x"^ < a 
for some constant a > 1. It is easy to check that the program above, together with this 
additional bound on the feasible set, satisfies the constraints qualification assumption (jH). 
For > 1, a lower bound In on the optimal value of the above problem is then computed 
as: 

In := inf X]"=il/i 

s.t. MnIv) h 

Mn-i {xi{vi - l)y) = , . 

Mn-1 {{a - x^x - v^v)y) >z ^ ' 

MN-i{viy) yO, forz = l,...,n 
Mn-1 {{ajx - bj) y) h 0, for j = 1, m, 

in the variable y E R^". Theorem 3.2 in |Las02j then states that there exists some A^* such 
that 

= MinCard(x), for all A^ > A^*, 
and the optimum is achieved whenever the rank of the matrices M^v ((...) y) stabilizes. 



4.2 The MinRank(X) problem 

In sectional for X G S", we saw that the optimum of the Min Rank(X) problem can be 
computed as the optimum value of the semialgebraic program: 

Era 

S.t. {v, - l)cr,{X) = 

Vi > 0, for i = 1, ...,n 

Tr (AjX) = bj, for j = 1, ...,p 

dj{X)>0, for/c[l,n]. 

To further simplify this program, we can substitute to the 2" constraints on the principal 
minors a more economical semialgebraic constraint. The modified program then reads: 

Era 
2=1 ^« 

S.t. (v, - 1)(T,(X) = 

Vi > 0, for i = 1, ...,n 

Tr (AiX) = bi, for i = I, ...,p 

u^Xu > 0, for u e R", 
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and again, to ensure compactness, we impose X'^X + v'^v + u^u < a for some constant 
a > 1. If we set the variable x = {u, X, v), for > [^^] , a lower bound Ijy on the optimal 
value of the above problem is computed as: 

In ■■= inf 

s.t. M^(y) y 

{{v,-l)a,iX)y) = 

Mm-i (v,y) y (14) 

M^_i((Tr {AX)-k)y) = 

Mn-1 iia - X^X + v^v + u^u) y) ^ 

for i = 1, . . . , n and j = 1, . . . ,p, in the variable y G R^", where the matrices M{q{x)y) are 
computed as in (fTT|) . Theorem 3.2 in |Lasn2j then states that there exists some such that 

In = Min Rank(X), for all N>N\ 

and the optimum is reached whenever the rank of the matrices Mn {{■■■) y) stabilizes. Alter- 
natively, one could use the fact that if we note Xt{X) the characteristic polynomial of X, 
then X ^ is equivalent to x~t^{^) being SOS as a univariate polynomial in t. 



4.3 Convex envelope 

Suppose that instead of having only one Min Card(a:;) or MinRank(X) problem to solve, 
we need to solve a (long) sequence of these problems with only some variation in the con- 
straints. Here, instead of computing an exact relaxation for every instance of the problem, 
we are interested in finding an efficient heuristic method for approximating the solution to 
all the problems to be solved. The complexity of the first "bound design" program will 
be high, but that of the subsequent programs will then be much lower. The heuristics in 
FHBOOj replaced the Card(a;) (resp. Rank(X)) functions by their convex envelope on 
the sets < x < 1 (resp. ^ X ^ J), i.e. the largest convex function f{x) such that 
f{x) < Card(a;) if < x < 1 (resp. /(X) < Rank(X) if ^ X ^ /). In this section, we 
extend these bounds to semialgebraic sets with more complex shapes. 

Of course, a function and its convex envelope share the same global minimum, so solving 
for this optimal lower bound is at least as hard as finding the global minimum. Here however 
we look for a convex lower bound for the problem in (|7j) inside the set of polynomials of degree 
at most d. This becomes a semialgebraic program: 

maximize J^^ ^-^^ p{x)dx 

subject to t — p{x) > on K , . 

Z)r=i ~ Pi^) >0 on K 
u^'V'^p[x)u > on = 1 

in the coefficients Pa of the polynomial p{x) G Rdfxi, where K is the compact semi- 

algebraic set given by: 

= (w, x) G R^" : {vi-l)xi = Q 

Ax-h>Q (16) 
X, f > 0. 
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Again, this can be cast as a LMI using the technique in |Las02j . We notice that the /i 
heuristic is a particular case when the constraint Ax — 6 > is dropped. 



5 Complexity 

Of course, the two semidefinite programs detailed in the last section are far from tractable 
if the dimension n and the relaxation order N grow beyond textbook example sizes. The 
Min Card(a;) problem is equivalent to solving 2" linear programs, so it is right to ask whether 
the programs above provide any benefit over, for example, branch-and-bound methods? 

Even if these two methodologies have similar worst-case complexities, the semidefinite 
relaxations in (fT!^ and ()14|1 do sometimes produce the global optimum for low order N (see 
|Lasn2j ) and because the objective is integer valued here, they only need to be solved up to 
an absolute precision of 1/2. We quickly detail below some other possible simplifications. 

But the results above have to be considered first as representations, providing an insight 
of the relative complexity of minimum cardinality problems versus that of tractable convex 
optimization problems. 

5.1 Structure, sparsity and symmetry 

The first element that can be used to simplify the programs in (fT^ and ()14|) is structure. 
In jLas02j for example, the constraints x G { — 1, 1}, that translate into x"^ — x = and 
— x)y) = also imply that the variables for a G Z^, can be replaced by ycard(a) 
in the program. Secondly, if the constraints in (jl3|) and (jl4|) include some symmetry, we 
could use the results in |GatOOj and |GP02j to preprocess and simplify the original semialge- 
braic program. The simplest example of these symmetries is of course when the constraints 
are invariant with respect to a change of basis, in which case MinRank(X) reduces to a 
MinCard(a;) problem and as in |GP02j p. 31, the asymptotic complexity goes from being 
exponential in n to being exponential in ^/n. 

In general, the complexity of algorithms in semialgebraic geometry grows at least ex- 
ponentially with the dimension. Efficiency is then very often measured by the ability of a 
method to maintain sparsity. Some results on Newton polytopes and SOS polynomials can 
then be used to efficiently handle sparsity in the SOS representations. In particular, a result 
of |R,ez78j shows that if p{x),hi{x) G tl[x], for i = l,...,r, with p{x) = Y7i=i^'ii^)^ then 
C{hi) C ^C{p). That result however does not hold as is for the representation in 

Finally, a lower bound on the optimal value can be obtained by simply dropping some of 
the constraints in (fT^ and (HH). 

6 Conclusion 

One of the central contributions of semidefinite programs to the optimization toolbox is 
their ability to efficiently solve a wide class of convex eigenvalue problems. In this work, 
we have illustrated how the method described in |Las01j for solving semialgebraic programs, 
by lifting them to semidefinite programs, can also be used to represent some semialgebraic 
eigenvalue problems and convex envelope relaxations. This contribution is centered around 
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semidefinite representations and the insight they can provide on the theoretical complexity 
of these problems. Wether or not they also improve the practical complexity of computing 
relaxations to these problems remains to be explored. 
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